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ABSTRACT 


The first of two related pamphlets, this guide 


provides information on how to conduct cost-outcome analyses, with an 
emphasis on meaSuring costs. After an introduction that delineates 
the purpose of the two-part series, the pamphlet is divided into six 


major sections. The first section, "definitions and rationale," 
introduces cost-outcome analysis and describes the differences 
between four primary types of cost-outcome analyses. The second 
section, "assessing political and pragmatic readiness," is designed 
to help determine the readiness of organizations for a cost-outcome 
analysis. The third section, "measuring program costs," explains how 
to collect resource data and how to figure costs, while the fourth 
section discusses assessment of the reliability and validity of the 
cost data. The fifth section highlights possible uses of the data in 
terms of cost feasibility analysis. Two appendixes are included: an 
example of listing of resources, and a sample cost distribution 
chart. (TE) . 
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In this world three things are certain: Death, 
taxes, and program budgets considered too small by 
those conducting the program. 
Poor Halcolm's Almanac (Patton, 
Practical Evaluation, p. 99.) 


PURPOSE OF THIS COST-OUTCOME GUIDE SERIES 


~ 


“This guide series was developed in response to evaluators’ 
reported need for information on how to conduct cost-outcome 
analyses. A recent study on the use of cost-outcome analyses by 
state education agency (SEA) evaluators (Smith, N. L. & Smith, J. 
K., 1984) and local education agency (LEA) evaluators (Smith, J. 
K. 1984) showed 


ls in the next five years, 60 percent of all SEAs 
expect to be asked to conduct cost-outcome © 
analyses, 


2a 71 percent of metropolitan LEAS expect to be 
asked to conduct cost-outcome analyse&, and 


3. one of the primary impediments to the conduct of 
cost-outcome analyses is the absence of useful 
guides and resources. 


To assist these and other evaluators in conducting cost- 
outcome analyses, two "how-to" guides were developed. Although 
several texts currently exist on cost-outcome anaysis, they tend 
to be written in technical language or do not give sufficient 
attention to collection of outcomes as well as costs. This guide 
series Supplements the existing texts by providing concise, 
readable explanations on how to conceptualize and conduct:cost- 
Outcome studies for program evaluation. 


This first guide introduces four types of cost-outcome . 
analysis, directs the collection of cost data, and explains how 
to select the most appropriate cost-outcome anlysis to answer an 
evaluation question. The second guide describes how to design an 
outcome study and outlines procedures for collecting outcome a 


data. 


When Should This Cost-Outcome Guide Series Be Used? 


Cost-outcome analyses can be applied ina variety of contexts 
to answer many different evaluation questions. The procedures 
are appropriate for evaluating mental health, health, and 
education programs located in community mental health centers, 
hospitals, schools, businesses, or any type of service proyram. 


Because of the wide range of possible applications, it was 
necessary to focus these guides on certain applications. This 
guide was written specifically for those evaluation studies that 


e are conducted at the local or state level, 
® use experimental or quaSi-experimental methods, 
e are conducted to compare two or more programs, 


e measure program outcomes. 


The guides are intended to assist the state or local level 
evaluator conduct a cost-outcome analysis for the purpose of 
programmatic improvement. For example, those interested in 
conducting a large-scale national study should consult the 
reference list of this guide for more technical information (see 
Levin, 1983; Sugden & Williams, 1978; Thompson, 1980). 


Introduction to This Guide 


Thys guide is organized into six major sections. The section 
entitled "Definitions and Rationale" introduces cost-outcome 
analysis and describes the differences between four primary types 
of cost-outcome analyses. The second section, "Assessing 
Political and Pragmatic Readiness," is designed to help determine 
the political and practical readiness of organizations for a 
cost-outcome analysis. The third section, "Measuriny Program 
Costs," explains how to collect resource data and how to figure. 
costs, while the tourth section discusses the assessment of the 
reliability and validity of the cost data. The tiftth section 
highliyhts possible uses of the data in terms of cost feasilibity 
analysis. 


DEFINITIONS AND RATIONALE 


The discussions throughout this guide are organized under a 
series of questions. These are tne questions most frequently 
asked by participants.in cost-outcome analysis workshops, or by 
evaluators seeking ccheut eattan on cost-outcome analysis. 


What Is a Cost-Outcome Analysis? 


Cost-outcome analysis systematically combines proyram outcome 
data with program cost data in such a way as to allow meaningful 
comparisons between two or more proyrams, Indeed, a cost-outcome 


analysis is simply an evaluation study which collects both 
program cost and outcome data using methods which are very 


familiar to evaluators. 


The programs ,being evaluated are rank-ordered according to 
their cost-outcome ratios. This rank ordering, along with other 
relevant information, can be used to facilitate decision making 
about programs. A schematic’ of procedures involved in a cost- 
outcome study is shown in Figure l.~ 


Sof Figure l 
Steps in Conducting a Cost-Outcome Analysis 
Identify program goals 
Select objectives to measure 


Determine most appropriate cost-outcome analysis 


s 
Select resources Select outcomes 
to measure ik to measure 
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Collect and Collect and 
tabulate costs ‘ analyze outcomes 


: N 
Combine cost data 
with outcome data 


v 


Interpret results 


How _is Cost-Outcome Analysis Different From 
Cost-Effectiveness or Cost-Benefit Analysis? 


The term--cost-outcome analysis--describes a group ot four 
different analyses which differ only with respect to the type of 
outcome measure collected. One method, cost-feasibility 
analysis, does not collect outcome data. This method 1S included 
in the guide because our research has shown it to ve the most 
widely applied cost-analysis method. Further, each cost-outcome 
analysis includes a cost-feasibility assessment, since cost- 
feasibility merely refers to the collection of cost data for 


comparison to budgetary constraints. The other three analyses 


build upon the cost-feasibility analysis by adding outcome 


measures. 


r 
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e Cost-utility analysis compares the cost of the 


program with 


The three other cost-outcome analyses are as follows: 


- the outcomes ot programs which are estimated 
rather than directly measured. 


e Cost-effectiveness analysis compares the cost of 


the programs with 


- the outcomes of programs aS measured in 
standard effectiveness units, such as test 


scores Or 


Satisfact ton 


indices. 


C Cost-benefit analysis compares the cost of the 


program with 


- the outcomes of programs as measured monetary 


units, 


such aS increase in income. 


Figure 2 displays the major differences amony the four 
methods in terms of the outcome measures they use and the 


comparisons they facilitate. 


Figure 2 


Differences Among Outcome Measures Used 
in Cost-Outcome Analyses 


Unit of 


Typical 


Enables 


Analysis Outcome MeaSure Outcome Measures Comparisons Vt 


Feasibility Se St eS 


Utility estimates 

Benefit monetary values 

Effective- effectiveness 
ness measures 
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experts' ratings of 
effectiveness; 

effectiveness of 
similar program; 


increases in 
income; 

value of life 
saved; 


test scores; 

behavioral ratings; 

satisfaction 
indices; 


all types’ of 
proyrains 


programs with 
ditferent 
goals (e.g., 
reading vs. 
math) 


programs with 
different 
goals (e.g., 
reading vs. 
math) 


programs with 
same or 
similar yoals 
(e.g., CAI 
reading vs. 
peer tutor 
reading) 


Our research has revealed many articles claiming the conduct 
of cost-effectiveness and cost-benefit analyses, when actually 
some other type of cost-outcome study, such as cost-feasibility, 
was conducted. In order té minimize confusion about the 
differences between the four types of cost-outcome analyses, we 
will refer to the yeneral class of procedures aS cost-outcome, 
and to the specitic methods by their appropriate names. 


What Kinds of Evaluation Questions Can Cost- 
Outcome Analysis Answer? : : 


Cost outcome analysis can provide answers to many evaluation 
questions posed by the sponsors of an evaluation (@.g., 
administrators, coordinators, participants, community, 
legislature). Listed below are just a few examptes ot the many 
types of questions cost analysis can provide answers to. 


How much does it cost to produce a unit of outcome in / 
Program A versus Program bB? 
UL 
Which program gives the most outcome for the least 
money 2 
e 
Will Program ‘A cost more to run in 3 years than would 


Program Bb in the same time? 


How Difficult Is It to Conduct a 
Cost-Outcome Study ? 


e 


In applied evaluation, it is always a challenge to conduct un 
evaluation that is both valid and reliable. Fortunately, 
measuring cost in addition to outcome data does not necessarily 
complicate that process. Cost information can be readily 
available in some cases, and all that is required is to complete 
a table that identifies program resources, and then to figure the 
cost of the relevant resources.: All of the calculations 
described in this guide can be wasily done with a pencil and 
paper. In most cases, it 15 tar more difficult to obtain 
celiable and valid outcon- data than it is to obtain reliable and 


valid cost data. 
»_ 


How Long Does It Take to Conduct 
a Cost-Vutcome Study? 


This question might be better worded as "How much longer does 
it take to conduct a cost-outcome study than just an outcome 
study?" The answer to this question depends upon many variables, 


including the size of the program, the support ot the staft, thr 
state of the financial records, and the time span within which 
the costs will be figured. Given these variables, Levin (1983) 
suggests that even the simplest cost-effectiveness or 
cos€-benefit study may take six months to conduct. 
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How Can You Place Monetary Values on the Types of 
Outcomes Usually Produced by Social Service Proyrams? 


This iS a concern often voiced by people who do not 
understand. the variety ot cost-outcome metnods available tu the 
evaluator. Only one method--cost-benefit analysis--attempts to 
place a monetary value on program outcomes. In cost- 
effectiveness and cost-utility analyses, outcomes are measured or 
estimated in standard units ot etfectiveness, such as test scores \ 
Or behavioral meaSures. These effectiveness measures are then 
compared to the cost of the programs. With cost-effectiveness 
and cost-utility analyses, social service outcomes are not 
translated into monetary terms. 


This section has introduced the types of cost-outcome 
analysis most frequently used by evaluators. In addition, 
answers have been provided to questions that evaluators commonly 
ask about cost-outcome analysis. The next section talks about 
political and pragmatic issues that affect the conduct of wu cost * 
analysis. ° 
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ASSESSING POLITICAL AND PRAGMATIC READINESS 


There are two categories of tactors which should be 
considered prior to conducting a cost-outcome analysis. The | 
tirst category 1S political readiness and the second is pragmatic 
readiness. The following two questions address these issues. 


What Political Factors Should I Consider 
Prior to Conducting a Cost-Outcome Study? 4 


rogram or Organization ready tO Support a 

tudy, and then to consider the data when making 

e are two specific political factors to consider 
question: (1) the attitudes ot the statt 

of the study, and (2) the potential 

study results. 

\ 


Is your 
cost-outcome 
decisions? Th 
in answering thi 
toward the conduc 
utilization otf th 


With respect to the first factor, you will need access to 
both the cost and outcome data. In some cases, this will require 
the assistance of other staff for direction and interpretation. 
Consider the support of the y 


e program staff, ; 
@ financial or accounting staff, 


e administrative staff. 


You also need to think about elements affecting the potential 
utility of the study results. These may be overriding conditions 
which severely limit the use of the cost-outcome data for 
decision making. If these conditions are present, then it may 
not be worth the effort to conduct the study. Some of these 
conditions are: 


e negative attitudes of decision makers toward use 
of quantitative cost and outcome data, 


e legislative or mandated regulations requiring 
program continuation, , 


®@ ethical constraints or Community reaction impediny 
program chance. 


What Pragmatic Factors Should I Consider 
Prior to Undertaking a Cost-Outcome Analysis? 


-Even in model programs, the collection of any type of data 
can be difficult, since data collection efforts may be impeded by 
time constraints, financial ceilings, ethical considerations, and 
methodological requirements. Therefore, before proceeding with a 
cost study, consider carefully the availability of the following: 


cost data, 

outcome data, 

staff to conduct the study, 

administrative support in terms of resources, 
experts for consultation or advice. 


If the above factors and conditions are favorable, then 
collection of cost data can begin. The next section explains how 
to collect cost data. 


MHASURING PROGRAM COSTS 
a 


What are the Preliminary Steps in 
Measuring Program Costs? 


The two steps to be taken before collecting cost data are as 
follows: 


ls Select a time frame for the study. 


2. Establish whether resources reflect start up or 
operation costs. 


| 
/ 

The first step in measuring program c ts is to select a time 
frame for the study. In most cases, the .Amount of time for the 
study will define itself naturally, based/ on the duration of the 
Programs, and will not exceed a year. If, however, the study 
exceeds one year, then Special adjustments must be made to the 
cost estimates. These adjustments are described in detail in 
this section. The special cases where the time span of the study 


may exceed one year include those times when 


. 


e a decision is to be made about two or more 
programs with payment schedules which differ over F 
the course of time, 


e the costs are calculated over the multi-year 
lifespan of the program. 


It is important to Specify a time span for the study prior to 
collection of any data, Since special adjustments on cost figures 
are necessary for studies with time spans that exceed a year. 
These adjustments are described later in this section. 


A second step in measuring program costs is to identify _ / 
whether the cost will be assessed for programs being considered 
for implementation, or for programs already in existence. The 
diffemgnce between these two situations will cause large 
differences in the. meaning of the cost figures, since the cost ot 
starting up a program is much higher tnan that of regular 
operation of a program due to larye expenditures at the onset. 
Although it would not be accurate to compare the cost of a 
proposed program along with its high start-up expenditures to 
that of a program currently running, an estimate ot the cost ot 
running the proposed program could be compared to the costs of a 


program currently tunning. 


Can You Use the Budget to 
Figure Program Costs? 


The purpose ot gathering cost duta In a Cost-outcome analysis 
is to 
l. provide a picture of what resources are required 


to run a program, and 
PAS calculate the cost of those resources. 


Although it would be convenient to use the budget, 'infortunately, 
budgets can be misleading indicators of program resources and 
their associated costs and are therefore not recommended for use 
in determining program costs (Levin, 1983). First, budgets may 
not list all program resources; second, the cost of th® listed 
resources may not be accurate; and third, the budget may not 
accurately reflect the typical cost of the program. 


) 
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With respect to providing a total listing of program 
resources, budgets typically do not include entries for resources 
such as: . 


e volunteer services, 
e borrowed equipment or materials, 


e materials, equipment, or facilities purchased in 
an earlier fiscal year. 


In addition, the costs tabulated in budgets are uSually based on 
Planned expenditures rather than on actual expenditures, and 
therefore may be inaccurate. Finally, since budgets. are figured 
On a yearly basis, they may be inflated spuriously by the 

_ purchase of equipment such as computers on a year-to-year basis. 
That is, one year the budget may be inflated by a purchase, while 
the next year it underestimates program cost, since no purchases 
were made. 


Why Worry About Resources That Are Not Included in the 
Budget Since They Obviously Do Not Add to Our Program Costs? 


bad 

It is important to generate a complete listing of resources, 
even though they do not represent direct costs for the program. 
One reason is that decision makers and evaluators alike need 
access to a total picture of the program resource requirements. 
For example, an examination of the resource list may point to the 
need for additional program equipment.: It may bring up questions 
about, the continuity of resources currently offered at no charge 
(e.g., volunteer services). In addition, if program outcomes are 
vastly different when compared, it may be useful to go back and 
examine all proyram resources that contributed to the production 
of the outcome. 


What Are Opportunity Costs? 


An opportunity cost is simply an alternative interpretation 
(not calculation) of program costs. The concept stems from the 
notion that societal support of a program represents sacrifice of 
resources that could theoreticaliy be used elsewhere. In order 
to reflect the opportunity costs to society for Supporting a 
program, all resources, including volunteers', students', and 
parents' time, should be valued monetarily and included in the 
total program cost estimate. All costs are opportunity costs, 
since theroetically the money could be used in an alternative 
manner. 
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HOW COSTS ARE FIGURED 


- 


Tabulation of costs is carried out in a two-step process. 
These steps are: 


1. Listing all the resources involved in running the 
program, and 


2. calculating the costs ot the appropriate resources. 


Step 1: Listing the Resources 


Appendix A contains an example of a list of program resources 
that may be used in a typical social service program. You will 
notice that the order of the list is from most significant 
(personnel) to least significant (supplies) resource_categories. 
The significance of a resource category is based upon e amount 
of money allocated to that category. For example, personnel 
costs usually represent 65 percent to 85 percent program costs, 
and so are very significant.’ In contrast, supplies usually 
represent only about 5 percent of costs and therefore are not 
significant. Although this example does not fully describe each 
resource, there should be enough detail about each listing to 
provide an accurate picture of the program resource 
requirements. You will notice that the list includes 
non-monetary as well as monetary costs. For example, volunteer 
time for student emotional reactions might be included. For an 
excellent discussion of non-monetary costs associated with 
testing programs, see Catterall (1982). 


This list can be generated through (1) interviews with 
program staff and administrators, (2) observations, and 
(3) reviews of records and files. Further, once the list is 
generated, it should be cross-referenced with the budget to 
insure that no resources have been omitted; and it should be 
reviewed by all relevant staff (e.g., program, financial, and 
administrators) for thoroughness. These checks help insure that 
the list provides a valid picture of program resources, an issue 
discussed more fully later in this guide. 
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Step 2: Determining the Costs of 


the Relevent Resources 


Determination of tne cost of resources is dependent on the 
anticipated use of the cost data. If the study results will be 
used within a particular district, then actual expenditures can 
be used as cost estimates. Haggart (1978) calls this 
“district-specific" costs. This guide assumes that the results 
of the analysis will not be disseminated outside the district. 


nL * 


For information on how to calculate costs that can be generalized 
to other programs, see Haggart (1978) for a description of 
“program-comparable” costs. 


“Each resource must be assigned a cost figure. Two: 
recommended methods for calculating the cost of a resource are: 


6 Market pricing--using the current market value of 
resource, . 


e shadow pricing--using the current market value of 
a similar resource. 


Levin (1983) suggests using these methods for valuing such 
categories of resources as the following. 


Personnel: If the current personnel are to be 
retained, then budget figures (which include fringe 
benefits) may be the best estimate. If new personnel 
need to be hired, then shadow pricing based on the 
‘going rate for staff with similar qualifications is 
appropriate. 


Facilities and equipment: Incorrect assessment of 
facilities and equipment. can either overestimate or 
underestimate program costs. Any equipment or 
facilities costs which were paid in previous years may 
not show up on the budget, but should be included in SN 
the cost of running the proyram, or the estimate will 
be too low. Similarly, the purchase of equipment or 

i facilities in a given year that is not typical will 
unfairly elevate the budget:for that given year. To 
estimate the value of equipment or facilities on an 
annual basi&, the recommended procedure is to: 


e obtain rental or.lease value from market sources 
(e.g., computer shop or real estate agent) or, if * 
this information is not available, 


@ calculate annual value by taking account of 
’ depreciation and interest on the undepreciated 
value (see Levin, 1983, p. 67). 


«- Supplieg: Supplies represent less than five percent 


ie of any ‘typical budget, and so do not affect cost 


‘egtimates* significantly. Therefore, for most 7 

i dnalyses, it is sufficient to use budget values or 

: even to omit happlies from the cost estimates, unless 
for sone, reason ENey represent a substantial cost. 


Client costs: Client costs are those incurred by 
program participants. Such costs can include lab 
fees, transportation charges, equipment costs. Use 
Market value of cost of transportation, equipment, 
books, or other items participants must purchase. 

) 


Appendix B provides an example of A tabulation of the costs 
for a computer-assisted instruction program and shows how the 
costs are distributed across several /jagencie#® Assessing the 
distribution of costs across agencieS is discussed in an upcoming 
question. The cost=per-student can/be obtained by dividing the 
total cost by the number of program participants. 


What About the Difference between the Costs of 
First Implementing Versus Actually Operating j 


A_Program? / 


If you are projecting the costs of a program not yet inplemented 
or calculating the costs of a first-year program, you should 
consider the difference between “start-up” costs and operation 
costs. When the program is being first implemented, its costs 
will be high due to the initial purchase of equipment and 
supplies. You can imaging that the purchase of one $20,000-bus 
would significantly elevate the program costs for a given year. . 
In a Situation Like thay the annualized value of the bus should 
983) provides a table which simplifies 


be calculated. Levin (¢/ 
alized value (see page 70). 


the calculation of an 


/ 
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What Adjustments Need to Be Mafle 
If The Time Span Exceeds a Year? 


‘ if a 
Three predominant reasons .to consider the cost’of a program 
over a period of several years are that one must 


take into account the effects of inflation over 
several years, 
¢ 


calcylate total costs for running a program over 
its Aife span, and 


1}. 


ure alternative program costs when payment 

schedules differ significantly. 
First, Gecision makers may be interested in determining whether 
they n afford to run the programs in five years. Generally, 
this Question is asked with respect to inflation increases. In 
cost-outcome analysis, however, results are intended to point to 
differences in ratio rankings between programs. Therefore, since 
inflation will presumably affect all costs equally, it should not 


Second, if a program were designed to run for three years, 
decisions makers may wish to know the total costs for all three 
years. This analysis may need to figure start-up costs, regular 
Operation costs, and the present value of resources which will 
not be paid for until later years. This concept otf present value 
is described below. 


Third, another reason to consider costs over several years 
would be to measure the differences between proyrams where each 
one has its own payment schedule. For example, one programemay 
require payment for equipment in the first year, while another 
May not require any payment for three years. The money invested 
for three years prior to expenditure will be earning interest and 
therefore growing. Economists talk about the "present" value ot 
‘money which will be spent in the future. The present value is 
calculated by subtracting the interest earned on the invested 
money from the total expenditure. For example, suppose Program A 
had to invest $1,000 today and Program B could defer that same 
expense for one year and earn 10 percent interest. Program A's 
actual cost is $1,000, while Program B's is only $900 
($1,000-$100) due to interest earnings. Levin (1983, pp. 96-98) 
describes how to calculate the present value of future 
expenditures. ‘ 


What If Staff, Equipment or Facilities 
Are Shared Across Several Programs? 


Obviously, it would be misleading to include the total cost 
for building rental on a program cost estimate if the program 
shares the building with three other programs. In cases like 
this, the joint cost is easily figured by computing a percentage 
of building rental based on the uSe by the program. Using a 
percentage is appropriate for figuring any cost that is shared or 
distributed across more than one program. 


Can You Determine the Stability of Funding 
Using Cost-Outcome Analysis? 


The stability of funding levels is a function of the funding 
sources. Basically, this refers to whether funding will be 
continued from .all sources at the same or similar levels over the 
next few years. Levin (1983) suggests an analysis of the 
distribution of funding sources as a means of assessing expected 
continuity of funding. This step breaks down costs according to 
support provided by the sponsoring agency, other government 
agencies, volunteers, and clients. This analysis is useful for a 
complete picture of program costs. An example is located in 
‘Appendix B. . eo ¢ 


f 
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Once the cost data is collected, it is important to evaluate 
the quality of that data prior to combining it with outcome 
measures.( If the cost data misrepresents the true costs of the 
program, hen. the cost-outcome ratios would be meaningless. The 
folgoPigig section suygests that the cost data be examined for 
reliaQility and validity. 
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*. ASSESSING RELIABILITY AND ®&ALIDITY OF THE COST DATA 
1 
Reliability and validity of data are familiar concepts to 

“evaluators, who routinely consider the accuracy and 
meaningfulness of data when designing analyses and interpreting 
results. Resources and cost data should be scrutinized in the 
same manner. In particular, the validity of the resource listing 
should be assessed, and the reliability of the cost estimates 
examined. 


How Should You Determine 


the Validity of The Resource List? 


External validity refers to the generalizability of data 
beyond the present study. The study results may be géneralizable 
to other programs for implementation or planning purposes. 
Ongoing programs, however, are uSually unique and so a resource 
list should not be assumed accurate for another ongoing program. 


Internal validity is important, since it refers to the degree 
to which the resource listing accurately describes the actual 
program. This is most relevant to meaningful internal decision 
making. To assess internal validity, the list of resources 
should be shared with program staff and administrators to insure 
that the listing accurately portrays the program resource 
requirements. Once it has been ascertained that the list is a 
valid reflection of the program, the reliability of the cost 
estimates should be considered. 


How Can You Tell If a Cost 
Estimate Is Reliable? 


Reliability refers to the stability of the cost figures, 
given the possibility of error in estimating the costs or changes 
in actual costs of the resources. The cost estimate for each 
resource should be examined because several small (or systematic) 
errors, when combined, can result in a total cost figure that is 
quite misleading. If these errors in measurement occur for 
resources representing a large percentage of the total cost, 


. 


15 16 


their effects can be profound. For example, a 5 percent error on 
the estimate of $500 for supplies would change the total cost by 
only + - $25. A 5 percent error on the estimate of $60,000 for 
personnel would change the total cost of + - $15,000. 


The reliability of cost estimates can be obtained by 
® having program staff evaluate the estimates, 


@ comparing specific resource costs with their 
budget figures, 


e comparing estimates with those derived in another 
manner (e.g., comparing lease values with annual 
values for facilities). 


What If The Cost Estimate 
Is Not Reliable? 


If there is any reason to suspect the accuracy of a cost ™s 
estimate, a sensitivity analysis should be conducted. A 
sensitivity analysis is simply a way of constructing confidence » 
intervals around a cost estimate. Given the range of po@Sible 
costs for a resource, “a low, medium, and high cost estimate can 
be derived. This procedure should be calculated for all resource 
estimates with questionable values. The three cost totals can be 
combined with outcome data. If, when two programs are compared, 
one program is more cost effective at the low, medium, and high 
levels than is the alternative program, you can be confident that 
it is a reliable analysis. Levin (1983) suyyests that a 
sensitivity analyses should be conducted if 


e new personnel will be hired at unset salaries, 
e a new facility is being constructed, 
8 the level of interest rate is questionable when 


calculating present values. 


Sensitivity analyses prpvide a means of dealing with uncertainty 
in cost estimates so that confidence in the cost aspect of the 
analysis can be achieved. 


After assessing the validity and reliability of the cost 
estimates, the information can be used to answer questions about 
Program resource requirements. Most often the total cost figures 
can be used to measure cost feasibility--whether the program or 
programs are affordable. However, information derived from all 
steps of the cost analysis can be useful, as shown in the next 
section. 


“1 


16 


DETERMINING COST FEASIBILITY 


Cost feasibility is the type of cost analysis most frequently 
conducted in evaluation today. The. purpose of conducting a cost- 
feasibility study is to determine whether it is feasible to 
either start up or maintain a program given financial 
constraints. The questions listed below demonstrate how the 
process of determining costs, as well as figuring the total 
program cost, can provide important information. For example, 
from listing resources you can answer these questions: 


Which resources are needed to start up a program? ie 
Which resources are needed to run a program? - 
Which resources might need to be replaced in the near 
future? 
Which resources are least essential to the program? 


From figuring the cost of each resource, you can answer these 
questions: ‘ 5 


What makes the program so expensive? 
What is the distribution of costs among resource 
categories? ; 
‘Where are we spending the most money? 
Where could costs be minimized? 


From figuring the total program costs, you can answer these 
questions: i 


What are the per-student costs? 
ae Which program can we afford? 


What Is the Next Step? 


Cost feasibility also forms the basis for other cost-outcome 
analyses, since these other analyses compare various types of 
outcome data with the cost data just described to form 
cost-outcome ratios. 


The next guide addresses the issues of selecting and 
measuring an outcome measure (e.g., utility, benefit, or 
effectiveness) to combine with the cost data in order to forma 
cost-outcome analysis. 
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APPENDIX A 


Example of Listing of Resources 


(1) Personnel 


coordinator (full-time) 
teacher aide (full-time) 
parent volunteer (half-time) 


oh 
oy 


(2) Facilities 


classroom--30 feet by 45 feet 
classroom furniture--20 desks and chairs 


(3) Equipment 


computers--10 Apple IIe total systems (includes printers) 
provisions for maintenance of equipment 


(4) Materials and Supplies 


curricula--software for mathematics and reading for 
grade 4 
supplies 


(5) Miscellaneous 


energy (heating, lighting, and power) 
routine maintenance of classroom 
administrative overhead 

insurance 

inservice training 

evaluation services 


(6) Client costs 


transportation 
disks 

7 two hours of parent time weekly 
six hours of student time weekly 
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APPENDIX B 


Distributing Costs (Levin, 1984) 


Cost to Imposed 
Other Contributed Student 
Pst Total Cost to Government Private and Family 
Ingredients Cost Sponsor Agencies Inputs Costs 
Personnel 
Coordinator $31,000 $31,000 
Aide 12,400 6,200 $ 6,200* 
Facilities 
Classroom 6,612 6,612 
Renovations 3,254 3,254 
Materials and 
Equipment 
Computer 17,250 17,250** 
Printer — 528 528 
Terminals 8,732 8,732 
Equipment 
Maintenance 1,852 1,852 
Curriculum 6,528 6,528 
Supplies 1,000 1,000 
Other 
Energy 1,500 1,500 
Routine 
- Maintenance 500 500 
Overhead 1,500 1,500 v 
Insurance '2,000 2,000 
Training 383 383 
Value of 
Client Time ; 
and Other 
Client Inputs 
~ eco 
Total Ingre- ’ i 
dients Cost $95,039 $72,589 $23,450. 
User Fees , v 
Other Cash 
Subsidies -20,000*** +21,000 + 1,000 
. Net Costs $95,039 $50,589 $20,000 $24,450. 
*volunteer. 
**Donated. 


***Subtraction based on other subsidies of $20,000 and $1,000. 
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